Skip to main content
Version: DAI 6.5

Working with Test Configurations

A test configuration is a combination of test cases and exploratory tests that can be run on demand or be scheduled to run on a regular basis in Eggplant DAI. A test configuration is essentially a blueprint of tests that can be run one after the other against a system under test (SUT) in a specific testing sequence.

Go to Controller > Test Config, to create, edit, or view your defined test configurations. The following columns are displayed to provide at-a-glance details about each test configuration:

  • Name: Display name of the test configuration.

  • Model/Suite: Name of the model or suite depending on the type of test configuration you've chosen to run.

  • Description: Description of the test configuration. Provide a meaningful description to help distinguish it from other test configurations.

  • Tags: Tags defined for the selected test configuration that you're viewing.

  • SUT: Associated SUT, or criteria for choosing a SUT for the test configuration.

  • Updated: Date and time when the test configuration was last updated.

  • Steps: Number of test steps included in the test configuration. A test step is either a test case from the model or a script based test case, or an exploratory test with stop criteria, which could be defined by duration, iterations, or a target coverage percentage.

Click the Actions Actions menu icon menu to Edit or Delete a test configuration.

To filter test configurations, type part of a test configuration name in the Search field or if you know the name of the model, select it from the All Models drop-down list.

Creating Test Configurations​

Go to Controller > Test Config to create tests that can be run on a regular basis. You must specify at least one SUT against which the test will be run and associate it with a model.

note

For exploratory steps, a stop criterion is always set by duration, iterations, or target coverage. After the test runs out of steps, Eggplant DAI will examine the execution of each step and indicate whether the test configuration is a success or failure.

Step by Step: Creating a Test Configuration​

Follow these steps to add a test configuration:

  1. Go to Controller > Test Config.

  2. From the top menu, click Add New +.

  3. Enter the Name and Description for the test configuration so that you can easily identify it in your existing list of test configurations.

  4. Add Tags by entering a name for the test configuration tag and pressing Enter. You can add as many test configuration tags as you need.

  5. Specify the SUT selection for the test to run on by selecting the appropriate options:

    • SUT by Name to select a predefined SUT.
    • SUT to use based on criteria to define one or more SUT criteria from the list. All the SUTs that are defined on the SUTs tab are available in this drop-down list. The SUT criteria field is similar to Test configuration tags, i.e., if you enter a SUT criteria and press Enter, relevant values are populated.

    For Cloud users, the options are slightly different:

    • Execution Environment: Select from the options below to determine whether you want to use DAI Eggplant Cloud Execution Environment including the Agent and EPF with Cloud SUTs or your local SUTs defined in the SUTs tab. Select Cloud to execute tests on Cloud SUTs. For using the Cloud SUT in Design mode, see Connecting to DAI Cloud SUTs.

    Cloud execution environment options

    • Local: Lets you select a predefined SUT or define a SUT based on criteria. To define a SUT based on criteria, enter a SUT criteria and press Enter.

    • Cloud: Enables the Eggplant Cloud Hosted SUT option:

      • Eggplant Cloud Hosted SUT (run on Chrome or Firefox): Select this to execute tests on Cloud SUTs where Eggplant will provision the Cloud Execution Environment (including the Agent and Eggplant Functional) for you. For more information, see Connecting to DAI Cloud SUTs.

      To execute tests using Cloud Execution Environment and your own local SUTs, you must ensure that your firewall rules allow connections from the Cloud Execution Environment. All connections from the Cloud Execution Environment to the SUTs will come from one of these fixed IP addresses:

      • 54.195.73.56
      • 54.170.198.89
      • 54.170.92.66

  6. Select the Type of test configuration you want to run. The selection you make here will determine the steps to add to your test configuration.

    Test configuration types

    • Script Based: Enables the option to select a suite and then add script based steps. Note that scripts are only be available if you have at least associated one suite in your Model tab. For detailed steps, see Using Script Based Test Configuration.
    • Model Based: Enables the option to select a model in order to run model based test configuration.

  7. For Model Based test configuration, select a model, then add an exploratory step or directed test cases, or both.

    • To add an exploratory step, select Add Exploratory and specify a target for the exploratory test steps by selecting one of the following options:
      • Iterations: Determines the number of times that the exploratory test should execute before moving on to next step in the test configuration. The test configuration will end if there is no other step left in the list of tests. By default, the iteration value is set to 10.
      • Coverage: Determines the minimum coverage required to complete a step. In Eggplant DAI, coverage represents the percentage of states and actions visited in your model during a series of exploratory test runs. Note: A test will pass even when the coverage doesn’t meet the minimum target coverage. For e.g., the test will stop when the target coverage is not achieved within a reasonable time, or until the point at which coverage stops increasing.
      • Time: Select to specify the duration of the exploratory step in hours.
    • To add specific directed test cases to your test configuration, select from the list. Note that test cases will only appear if you have created at least one directed test case in the Test Cases tab in Modeler.

  8. Click Save. The new test configuration is saved to the list of test configurations in the Test Config tab.

Tips
  • Reorder your steps by clicking the Drag icon Drag icon, then drag and drop steps as required.
  • Delete your steps by clicking the Delete icon Delete icon.

Running a Test Configuration​

You can run a test configuration from Controller > Runner by clicking Play button icon against the test you want to run.

Go to Controller > Runner to configure test schedules with specific frequencies to be run at regular intervals. You can only have one schedule per test configuration.

tip

Remember to always associate suites to a model before running tests.

note

When you abort a test run that is part of a test configuration schedule, all runs that are part of that configuration schedule are aborted.

Viewing, Editing and Notification Settings of an Existing Test Configuration​

To view or edit a test configuration:

  1. Go to Controller > Test Config.
  2. Find the test configuration that you're interested in updating or viewing, then from the Actions menu icon Actions menu, select Edit.
  3. From here you can view or update the test configuration details.
  4. Furthermore, you can also manage the notification settings of the test configuration by clicking on link.

Notification Settings of an Existing Test Configuration​

Important

Test Configuration notifications must be enabled globally before you can set up notifications for a specific Test Configuration within DAI. You need to be an administrator to do this. See Manage DAI Settings for details.

By default, the notification feature for all Test Configurations is disabled. To enable it, follow these steps:

  1. Go to Controller > Test Config.
  2. Find the test configuration that you're interested in, then from the Actions menu icon Actions menu, select Edit.
  3. Click on link.
  4. From here, you can enable / disable the feature and assign verified users to receive the notifications.
  5. These are the rules for this feature:
    • If the feature is enabled, then you have to pick at least one recipient before you are allowed to save.
    • If the feature is disabled, then you can leave the field blank and save.

Removing a Test Configuration​

To remove an existing test configuration:

  1. Go to Controller > Test Config.
  2. Find the test configuration that you want to delete, then from the Actions menu icon Actions menu, select Delete.
  3. From here you can view or update the test configuration details.

Duplicating an Existing Test Configuration​

To duplicate an existing test configuration:

  1. Go to Controller > Test Config.
  2. Find the test configuration that you want to duplicate, then from the Actions menu icon Actions menu, select Duplicate.
  3. From here, you will be required to enter a new name for the Test Config.
  4. Click Create once a valid name is entered.
  5. From here, you will be able to edit your newly-duplicated Test Config.
Tips
  • Make sure to avoid entering the same name as the Test Config that is to be duplicated.
  • The new name has to follow these rules:
    1. 2 to 100 alphanumeric characters.
    2. Space and dashes are accepted, however they cannot be used at the start or end of the name, only between other alphanumeric values.
    3. No other special characters are allowed.
  • Examples of acceptable names:
    1. New Test Config
    2. New-Test-Config
    3. New-Test
    4. New Test 2
Last updated on